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SUBJECT:  Audit  Report  on  Contractor  Selection  for  the  Development  of  the  Range 
Rule  Risk  Methodology  (Report  No.  98-026) 


We  are  providing  this  report  for  information  and  use.  This  audit  was  requested 
by  Congressman  Robert  E.  Cramer  who  asked  for  a  review  of  Government  use  of  the 
Ordnance  and  Explosives  Cost-Effectiveness  Risk  Tool  and  contractor  selection  for  the 
development  of  the  Range  Rule  Risk  Methodology.  Because  this  report  contains  no 
recommendations,  no  written  comments  on  a  draft  version  were  required,  and  none 
were  received. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  For  additional 
information  on  this  report,  please  contact  Mr.  Joseph  P.  Doyle,  Audit  Program 
Director,  at  (703)  604-9348  (DSN  664-9348)  or  Ms.  Deborah  L.  Culp,  Audit  Project 
Manager,  at  (703)  604-9335  (DSN  664-9335).  See  Appendix  B  for  the  report 
distribution.  The  audit  team  members  are  listed  inside  the  back  cover. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 
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Contractor  Selection  for  the  Development  of 
the  Range  Rule  Risk  Methodology 


Executive  Summary 


Introduction.  This  audit  was  requested  by  Congressman  Robert  E.  Cramer  who  asked 
the  Inspector  General,  DoD,  to  review  Government  use  of  the  Ordnance  and 
Explosives  Cost-Effectiveness  Risk  Tool  and  contractor  selection  for  the  development 
of  the  Range  Rule  Risk  Methodology. 

The  U.S.  Army  Corps  of  Engineers,  Huntsville  Engineering  and  Support  Center, 
Huntsville,  Alabama,  awarded  QuantiTech,  Inc.  a  contract  to  develop  the  Ordnance 
and  Explosives  Cost-Effectiveness  Risk  Tool.  QuantiTech,  Inc.  was  to  develop  a 
rational,  analytical  methodology  for  ranking  ordnance  ranges  for  remediation  of 
unexploded  ordnance  based  on  life-cycle  cost  and  programmatic  and  public  risks. 

The  DoD  Explosives  Safety  Board  is  responsible  for  the  development  of  the  DoD 
Range  Rule.  The  U.S.  Army,  specifically  the  Army  Environmental  Center,  Aberdeen 
Proving  Ground,  Maryland,  is  the  designated  lead  agent  for  drafting  the  rule.  The 
Range  Rule  will  be  the  DoD  regulation  that  specifies  a  process  for  initiating  and 
conducting  response  actions  on  closed,  transferred,  and  transferring  military  ranges. 

The  Range  Rule  Risk  Methodology  (part  of  the  DoD  Range  Rule)  currently  being 
developed,  will  establish  a  standardized  technical  method  to  evaluate  ranges  and  to 
determine  what  responses  are  necessary  to  protect  human  health  and  the  environment. 

Audit  Objective.  The  audit  objective  was  to  determine  whether  the  Government 
appropriately  used  the  Ordnance  and  Explosives  Cost-Effectiveness  Risk  Tool.  We 
also  reviewed  the  justification  used  to  select  the  contractor  to  develop  the  Range  Rule 
Risk  Methodology.  The  audit  was  also  to  review  the  management  control  program  as  it 
applied  to  the  other  objectives.  We  did  not  review  the  management  control  program 
because  the  scope  of  the  audit  was  limited  to  the  two  contracts  mentioned  in  the 
congressional  request.  See  Appendix  A  for  a  discussion  of  the  audit  process  and  prior 
coverage. 

Audit  Results.  We  determined  that  the  Government  appropriately  used  the  Ordnance 
and  Explosives  Cost-Effectiveness  Risk  Tool.  The  contract  for  the  development  of  the 
Ordnance  and  Explosives  Cost-Effectiveness  Risk  Tool  gave  the  Government  unlimited 
rights  in  technical  data  and  software  for  all  items  developed  with  Government  funds. 
Also,  the  Government  did  not  give  away  the  contractor’s  intellectual  property.  We 
determined  that  contractor  selection  for  the  development  of  the  Range  Rule  Risk 
Methodology  was  consistent  with  applicable  acquisition  rules.  See  Part  I  for  a 
discussion  of  the  audit  results. 

Management  Comments.  None  required. 
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Audit  Background 

Congressman  Robert  E.  Cramer  requested  that  we  review  the  Government’s 
rationale  of  providing  the  Ordnance  and  Explosives  Cost-Effectiveness  Risk 
Tool  (OECert),  developed  by  a  Small  Business  Administration  certified  8(a) 
firm  under  a  Government  contract,  to  another  contractor  for  its  use  and  further 
development.  An  8(a)  firm  is  defined  as  one  that  is  owned  and  controlled  by 
socially  and  economically  disadvantaged  individuals.  Congressman  Cramer  also 
requested  that  we  review  the  contractor  selection  process  for  providing  technical 
support  for  the  development  of  the  Range  Rule  Risk  Methodology  (R3M). 

OECert.  QuantiTech,  Inc.  developed  the  OECert  on  a  contract  with  the  U.S. 
Army  Corps  of  Engineers,  Huntsville  Engineering  and  Support  Center, 
Huntsville,  Alabama.  QuantiTech,  Inc.  was  to  develop  a  rational,  analytical 
methodology  for  ranking  ordnance  ranges  for  remediation  of  unexploded 
ordnance  based  on  life-cycle  cost  and  programmatic  and  public  risks.  The  cost 
reimbursable  contract  was  awarded  April  1993  in  response  to  a  Broad  Agency 
Announcement.  The  contract  was  not  an  8(a)  set  aside,  and  QuantiTech,  Inc. 
was  not  8(a)  certified  until  January  1995.  The  final  version  of  the  OECert  was 
issued  in  August  1995.  The  Army  Corps  of  Engineers  provided  copies  of  the 
OECert  to  otiier  Government  agencies  who,  in  turn,  provided  the  model  to  other 
contractors  involved  in  unexploded  ordnance  projects. 

Range  Rule  Risk  Methodology.  The  purpose  of  R3M  is  to  set  forth  the 
techmcal  method  that  the  DoD  will  use  to  support  its  Range  Rule.  The  Range 
Rule  will  be  the  regulation  on  how  the  DoD  will  respond  to  ordnance  clean  up 
at  its  ranges.  The  R3M  is  being  drafted  by  the  Army  Environmental 
Center  (AEC),  Aberdeen  Proving  Ground,  Maryland.  PRC  Environmental 
Management,  Inc.  (PRC  EMI)  is  providing  technical  support  to  AEC  in 
response  to  a  Technical  Direction  Letter  issued  by  the  Naval  Explosive 
Ordnance  Disposal  Technology  Division,  Indian  Head,  Maryland.  The 
Technical  Direction  Letter  specified  a  level  of  effort  not  to  exceed  $145,000, 
however,  funding  was  subsequently  raised  to  $220,000. 


Audit  Objectives 

The  audit  objective  was  to  determine  whether  the  Government  appropriately 
used  the  OECert  developed  by  a  contractor.  Also,  we  reviewed  die  justification 
used  to  select  the  contractor  to  develop  R3M.  An  additional  objective  was  to 
review  the  management  control  program  as  it  applied  to  the  other  objectives. 

We  did  not  review  the  management  control  program  because  the  scope  of  the 
audit  was  limited  to  the  two  contracts  mentioned  in  the  congressional  request. 
See  Appendix  A  for  a  discussion  of  the  audit  process  and  prior  coverage. 
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Government  Use  of  OECert 


The  Government  appropriately  used  the  OECert  developed  by 
QuantiTech,  Inc.  The  contract  awarded  to  develop  the  OECert  gave  the 
Government  unlimited  rights  in  technical  data  and  software  for  all  items 
developed  with  Government  funds.  Also,  the  Government  did  not  give 
away  Ae  contractor’s  intellectual  property.  The  Government  has  the 
right  to  disseminate  the  OECert  model  to  any  person,  agency,  or 
organization  deemed  necessary. 


Use  of  OECert 

The  Army  Corps  of  Engineers,  Huntsville  Engineering  and  Support  Center, 
appropriately  disseminated  the  Government-owned  Ofcert  model  to  be  used  by 
others  for  Government  purposes.  The  Army  Corps  of  Engineers  furnished 
copies  of  the  OECert  to  other  Government  agencies  involved  in  unexploded 
ordnance  detection  and  remediation.  The  OECert  was  also  given  to  contractors 
involved  in  Government  funded  unexploded  ordnance  projects. 


Data  Rights 

The  contract  with  QuantiTech,  Inc.  for  the  OECert  gave  the  Government 
unlimited  rights  in  technical  data  and  computer  software  that  was  generated 
during  development  of  the  model.  The  OECert  contract  contained  the  “Rights 
in  Technical  Data  and  Computer  Software  (Oct  1988)”  clause.  The  clause 
states  that  the  Government  is  entitled  to  and  will  receive  unlimited  rights  in 
technical  data  pertaining  to  an  item,  component,  or  process  that  has  been 
developed  exclusively  with  Government  funds.  The  Government  also  has 
unlimited  rights  in  manuals  or  instructional  materials  prepared  or  required  to  be 
delivered  that  are  necessary  for  installation,  operation,  or  training  purposes. 

The  clause  also  gives  the  Government  unlimited  rights  to  computer  software 
developed  during  the  performance  of  the  contract.  The  final  OECert  report  and 
accompanying  software  did  not  contain  any  restrictive  markings  that  would 
indicate  that  the  contractor’s  product  contained  any  proprietary  data  or  software. 
The  Government  had  no  restrictions  on  the  dissemination  of  data  or  software 
developed  under  the  contract  with  QuantiTech,  Inc. 


Intellectual  Property 

The  Government  did  not  unfairly  give  away  any  intellectual  property  of 
QuantiTech,  Inc.  The  OECert  was  developed  using  standard  risk  evaluation 
techniques,  and  no  new  risk  evaluation  theories  were  developed.  U.S.  Army 
Corps  of  Engineers,  Huntsville  Engineering  and  Support  Center  personnel 
indicated  that  the  basic  premise  of  the  OECert  was  to  apply  operations  research 
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Government  Use  of  OECert 


modeling  concepts  to  the  area  of  unexploded  ordnance  risk  evaluation.  Corps  of 
Engineers  personnel  stated  that  there  were  no  new  theories  developed  or  applied 
in  die  OECert  model.  Further,  Office  of  Inspector  General,  DoD,  Quantitative 
Methods  Division  personnel  reviewed  the  final  OECert  report  and  concluded 
that  the  OECert  model  concept  is  based  on  relatively  standard  approaches  for 
computing  encounter  probabilities  and  area  sweep  rates.  The  idea  of  providing 
modified  equations  for  each  of  several  different  operations  (in  this  case, 
different  site  uses)  is  an  established  militaiy  operations  research  technique,  one 
that  is  applied  in  any  assessment  of  a  multimission  system.  Quantitative 
Methods  Division  personnel  further  stated  that  conceptually,  many  operations 
research  analysts  might  adopt  such  an  approach  to  the  problem.  The  general 
nature  of  the  conceptual  approach,  and  its  similarity  to  antisubmarine  warfare 
search  and  minefield  sweeping  problems,  would  seem  to  argue  against  the 
OECert  being  an  example  of  unique  intellectual  property. 


Summary 

The  Government  funded  the  total  cost  of  developing  the  OECert  model  so  it 
could  be  used  as  a  tool  for  the  remediation  of  unexploded  ordnance  at  military 
ranges.  The  Government  had  unlimited  rights  in  all  technical  data  and  computer 
software  developed  under  the  contract.  The  contractor  did  not  claim  that  it  had 
furnished  the  Government  any  proprietary  data  with  the  contract  deliverables. 
The  contract  under  which  the  OECert  was  developed  was  in  response  to  a  Broad 
Agency  Announcement,  open  to  all  competition,  and  was  not  a  small  business 
or  8(a)  set-aside.  At  the  time  of  the  award,  QuantiTech,  Inc.  was  not  8(a) 
certified  by  the  Small  Business  Administration.  Regardless,  8(a)  contractors  are 
not  granted  special  rights  or  privileges  in  the  retention  of  data  rights  for  which 
the  Government  has  paid.  The  QuantiTech  model  did  not  develop  or  apply  new 
theories,  instead  it  is  based  on  relatively  standard  approaches  for  computing 
encounter  probabilities  and  sweep  rates.  The  Government  acted  correctly  in  the 
distribution  of  the  OECert  model.  It  is  unreasonable  to  expect  restriction  of 
OECert  distribution  for  any  legitimate  Government  purpose. 
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Contractor  Selection  for  R3M 


The  Army  did  not  violate  the  Competition  in  Contracting  Act  by  placing 
an  order  with  the  Navy  for  technical  support  for  the  development  of  the 
R3M.  The  Navy  could  have  placed  an  order  for  the  task  of  providing 
technical  support  for  the  development  of  the  R3M  on  any  existing  open 
contract  with  a  scope  of  work  commensurate  with  the  work  to  be 
performed.  The  Navy  was  not  required  to  compete  the  award.  The 
Navy  correctly  awarded  the  work  to  provide  technical  support  for  the 
development  of  the  R3M  to  a  qualified  contractor. 


Contractor  Selection 

The  AEC  personnel  requested  the  Naval  Explosive  Ordnance  Disposal 
Technology  Division  (Navy)  to  provide  technical  support  for  the  development  of 
the  R3M.  The  Navy  had  an  existing  cost  plus  fixed-fee  level  of  effort  contract 
with  PRC  EMI  for  efforts  pertaining  to  unexploded  ordnance  clearance 
technology.  The  work  to  be  performed  on  the  R3M  was  within  the  scope  of 
work  of  the  PRC  EMI  contract.  The  Navy,  at  the  direction  of  AEC,  issued  a 
Technical  Direction  Letter  not  to  exceed  $145,000  to  PRC  EMI  to  provide  the 
technical  support  for  R3M. 


Extent  of  Competition 

The  work  effort  for  providing  technical  support  for  the  development  of  the  R3M 
was  not  openly  competed.  The  AEC  project  manager  indicated  that  he 
judgmentally  selected  a  Navy  contractor  (PRC  EMI)  to  perform  the  work  for 
the  R3M.  To  confirm  his  selection,  the  AEC  project  manager  subsequently 
prepared  an  internal  selection  matrix  on  which  he  ranked  PRC  EMI, 

QuantiTech,  Inc. ,  and  one  other  contractor  for  the  development  of  the  R3M. 
Matrix  factors  included  product  quality,  R3M  history,  model  knowledge  (Navy, 
OECert,  Ft.  Meade),  survey  methods  knowledge,  multiuser  integration,  multi¬ 
service  link-ups,  contracting  capacity,  availability,  costs,  AEC  history,  and 
procurement  cycle.  The  project  manager  stated  that  he  independently  prepared 
the  one-page  matrix,  and  did  not  solicit  information  firom  any  contractor 
representatives  or  Government  agents.  All  contractors  were  ranked  equal  on 
product  quality,  contracting  capability,  availability,  and  costs.  The  AEC  project 
manager  did  not  have  documentation  to  support  the  criteria  or  the  scores 
contained  in  the  matrix.  However,  based  on  his  internal  contractor  ranking 
matrix,  he  reconfirmed  his  decision  to  select  PRC  EMI  for  the  effort.  The  AEC 
transferred  funds  to  the  Navy  to  provide  technical  support  for  the  development 
of  the  R3M,  and  verbally  requested  that  PRC  EMI  perform  the  work.  Because 
the  PRC  EMI  contract  was  available,  and  the  scope  of  work  would  support 
development  efforts  for  R3M,  the  Navy  was  not  obligated  to  compete  Ae 
technical  effort  for  R3M. 
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Contractor  Selection  for  R3M 


Summary 

The  decision  to  award  the  task  of  providing  technical  support  for  the 
development  of  the  R3M  without  competition  was  correct.  Contracts  were 
available  for  development  of  R3M  without  having  to  obtain  full  and  open 
competition,  and  the  project  manager  judgmentally  selected  PRC  EMI  to 
perform  the  work.  The  work  efforts  for  the  task  were  within  the  scope  of  the 
contract  selected. 
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Part  II  -  Additional  Information 


Appendix  A.  Audit  Process 


Scope  and  Methodology 

Work  Performed.  We  reviewed  contract  DACA87-93-C-0036  for  the 
development  of  the  OECert.  This  contract  was  awarded  in  FY  1993  and  valued 
at  about  $1.1  million.  We  also  reviewed  Technical  Direction  Letter  0007-A 
issued  in  FY  1996  on  contract  N00174-96-C-0075,  valued  at  $145,000  and 
subsequently  increased  to  $220,000,  for  the  development  of  the  R3M.  We 
reviewed  contract  documentation  for  FYs  1993  to  1997.  During  the  audit,  we 
interviewed  DoD  personnel  involved  with  the  study  and  remediation  of 
unexploded  ordnance,  and  personnel  involved  with  procurement  and 
administration  of  the  reviewed  contracts.  We  also  reviewed  the  use  and 
disposition  of  contract  deliverables. 

We  did  not  review  the  management  control  program  because  the  scope  of  the 
audit  was  limited  to  the  two  contracts  in  the  congressional  request. 

Technical  Assistance.  Operations  research  analysts  from  the  Quantitative 
Methods  Division,  Analysis,  Planning  and  Techmcal  Support  Directorate  of  the 
Inspector  General,  DoD,  provided  technical  assistance  related  to  the  modeling 
concepts  contained  in  the  OECert. 

Audit  Period  and  Standards.  We  performed  this  economy  and  efficiency 
audit  from  May  through  October  1997  in  accordance  with  auditing  standards 
issued  by  the  Comptroller  General  of  the  United  States,  as  implemented  by  the 
Inspector  General,  DoD.  The  audit  did  not  rely  on  computer-processed  data  or 
statistical  sampling  procedures. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 


Prior  Coverage 

Office  of  Inspector  General,  DoD,  Report,  “Review  of  Policies  and  Procedures 
Guiding  the  Cleanup  of  Ordnance  on  Department  of  Defense  Lands,” 
November  1994.  The  evaluation  report  concludes  that,  among  other  things, 
ordnance  detection  and  removal  technologies  observed  were  simplistic,  labor- 
intensive,  and  primitive.  The  report  states  that  technology  is  needed  that  is 
quick  and  economical  in  identifying  and  removing  unexploded  ordnance  and 
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Appendix  A.  Audit  Process 


explosive  waste.  The  report  recommended  that  the  “Deputy  Under  Secretary  of 
Defense  (Environmental  Security)  should  direct  the  Department  of  Defense 
Explosive  Safety  Board  to  incorporate  risk-based  standards,  similar  [emphasis 
added]  to  those  being  developed  using  the  U.S.  Army  Corps  of  Engineers’ 
Quantitech  [sic]  Model,  in  their  unexploded  ordnance  cleanup  standards.”  No 
conunents  were  received  to  the  report. 
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Appendix  B.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 
Deputy  Under  Secretary  of  Defense  (Environmental  Security) 
Director,  Defense  Logistics  Studies  Information  Exchange 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Assistant  Secretary  of  the  Army  (Installations,  Logistics,  and  Environment) 
Assistant  Chief  of  Staff  for  Installation  Management 
Commander,  Army  Environmental  Center 
Auditor  General,  Department  of  the  Army 
Commander,  Army  Corps  of  Engineers 

Commander,  Huntsville  Engineering  and  Support  Center,  Huntsville,  AL 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 

Auditor  General,  Department  of  the  Navy 

Commander,  Naval  Surface  Warfare  Center,  Indian  Head  Division,  Indian  Head,  MD 
Coimnander,  Naval  Explosive  Ordnance  Disposal  Technology  Division, 

Indian  Head,  MD 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
Chairman,  Defense  Explosive  Safety  Board 


10 


Appendix  B.  Report  Distribution 


Non-Defense  Federal  Organizations 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 

House  Committee  on  National  Security 

House  Committee  on  Government  Reform  and  Oversight 

House  Subcommittee  on  Government  Management,  Iirformation,  and  Technology, 
Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 

Honorable  Robert  E.  Cramer,  U.S.  House  of  Representatives 
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Audit  Team  Members 


The  Contract  Management  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing,  DoD,  produced  this  report. 

Paul  J.  Granetto 
Joseph  P.  Doyle 
Frank  M.  Ponti 
Deborah  L.  Culp 
Carolyn  A  Gemand 
Michael  J.  Tully 
Dharam  V.  Jain 
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A .  Report  Title:  Contracting  Selection  for  the  Deveiopment  of  the  Range 
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